Jigs for use in medical imaging and methods for using thereof

ABSTRACT

A device for use during a medical imaging process, the device including a support structure and a plurality of radiopaque markers, the support structure configured to be positioned in proximity to at least a portion of a body of a patient during the medical imaging process, the plurality of radiopaque markers attached to the support structure, the plurality of radiopaque markers being positioned in a pattern such that an image capturing a given portion of the pattern is unique from an image capturing any other given portion of the pattern.

CROSS REFERENCE TO RELATED APPLICATION

This is an international (PCT) application relating to and claiming thebenefit of commonly-owned copending U.S. Provisional Patent ApplicationNo. 62/415,146, filed Oct. 31, 2016, entitled “METHODS AND SYSTEMS FORUSING MULTI VIEW POSE ESTIMATION,” the contents of which areincorporated herein by reference in their entirety.

FIELD OF THE INVENTION

The embodiments of the present invention relate to interventionaldevices and methods of use thereof.

BACKGROUND

Minimally invasive procedures such as endoscopic procedures,video-assisted thoracic surgery, or similar medical procedures can beused as diagnostic tools for suspicious lesions or as treatment meansfor cancerous tumors.

SUMMARY OF INVENTION

In an embodiment, a device for use during a medical imaging processincludes a support structure and a plurality of radiopaque markersattached to the support structure, the support structure configured tobe positioned in proximity to at least a portion of a body of a patientduring the medical imaging process, the plurality of radiopaque markersbeing positioned in a pattern such that an image capturing a givenportion of the pattern is unique from an image capturing any other givenportion of the pattern.

In an embodiment, the radiopaque markers include at least one of (a) aradiopaque metal or (b) a radiopaque plastic. In an embodiment, theradiopaque markers have a uniform shape. In an embodiment, the shape isselected from the group consisting sphere-shaped, rod-shaped, andcube-shaped. In an embodiment, the pattern includes one of (a)concentric circles or (b) a pattern of points arrayed along lines. In anembodiment, the pattern is generated using linear feedback shiftregisters. In an embodiment, the pattern is generated using XOR crossconnection of two one-dimensional linear feedback shift registergold-codes with different principal polynomials of the same order andN×M register stages.

In an embodiment, the support structure is substantially planar. In anembodiment, the support structure is three-dimensional. In anembodiment, the support structure is configured to be positioned above apatient's bed. In an embodiment, the support structure is configured tobe positioned below a patient's bed. In an embodiment, the supportstructure is configured to be positioned between a patient's bed and amattress positioned thereon. In an embodiment, the support structure isconfigured to be attached to a patient's chest. In an embodiment, thesupport structure is at least the size of a human chest.

In an embodiment, a method includes providing a jig including a supportstructure and a plurality of radiopaque markers attached to the supportstructure, the support structure configured to be positioned inproximity to at least a portion of a body of a patient during a medicalimaging process, the plurality of radiopaque markers being positioned ina pattern such that an image capturing a given portion of the pattern isunique from an image capturing any other given portion of the pattern;obtaining a first image from a first imaging modality; extracting atleast one element from the first image from the first imaging modality,wherein the at least one element comprises an airway, a blood vessel, abody cavity, or any combination thereof; obtaining, from a secondimaging modality, at least (i) a first image of the jig in a first poseof second imaging modality and (ii) a second image of the jig in asecond pose of second imaging modality, wherein the jig is positioned inproximity to a body of a patient; generating at least two augmentedbronchograms, wherein a first augmented bronchogram corresponds to thefirst image of the second imaging modality in the first pose, andwherein a second augmented bronchogram corresponds to the second imageof the second imaging modality in the second pose, determining mutualgeometric constraints between: (i) the first pose of the of secondimaging modality, and (ii) the second pose of the of second imagingmodality, estimating the first pose of the of second imaging modalityand the second pose of the of second imaging modality, wherein theestimation is performed using: (i) the first augmented bronchogram, (ii)the second augmented bronchogram, and (iii) the at least one element,and wherein the estimated first pose of the of second imaging modalityand the estimated second pose of the of second imaging modality meetsthe determined mutual geometric constraints, generating a third image;wherein the third image is an augmented image derived from the secondimaging modality which highlights an area of interest, wherein the areaof interest is determined from projecting data from the estimated firstpose and the estimated second pose.

In an embodiment, the mutual geometric constraints are generated by: a.estimating a difference between (i) the first pose and (ii) the secondpose by comparing the first image of the jig and the second image of thejig, wherein the estimating is performed using a device comprising aprotractor, an accelerometer, a gyroscope, or any combination thereof,and wherein the device is attached to the second imaging modality; b.extracting a plurality of image features to estimate a relative posechange, wherein the plurality of image features comprise anatomicalelements, non-anatomical elements, or any combination thereof, whereinthe image features comprise: patches attached to a patient, radiopaquemarkers positioned in a field of view of the second imaging modality, orany combination thereof, and wherein the image features are visible onthe first image of the radiopaque instrument and the second image of theradiopaque instrument; c. estimating a difference between (i) the firstpose and (ii) the second pose by using at least one camera, wherein thecamera comprises: a video camera, an infrared camera, a depth camera, orany combination thereof, wherein the camera is at a fixed location,wherein the camera is configured to track at least one feature, whereinthe at least one feature comprises: a marker attached the patient, amarker attached to the second imaging modality, or any combinationthereof, and tracking the at least one feature; d. or any combinationthereof.

In an embodiment, the method also includes tracking the jig for:identifying a trajectory, and using the trajectory as a furthergeometric constraint.

BRIEF DESCRIPTION OF THE FIGURES

The present invention will be further explained with reference to theattached drawings, wherein like structures are referred to by likenumerals throughout the several views. The drawings shown are notnecessarily to scale, with emphasis instead generally being placed uponillustrating the principles of the present invention. Further, somefeatures may be exaggerated to show details of particular components.

FIG. 1 shows a block diagram of a multi-view pose estimation method usedin some embodiments of the method of the present invention.

FIGS. 2, 3, and 4 show exemplary embodiments of intraoperative imagesused in the exemplary method of FIG. 1. FIGS. 2 and 3 illustratefluoroscopic images obtained from one specific pose. FIG. 4 illustratesa fluoroscopic image obtained in a different pose, as compared to FIGS.2 and 3, as a result of C-arm rotation. The bronchoscope (240, 340,440), the instrument (210, 310, 410), ribs (220, 320, 420), and the bodyboundary (230, 330, 430) are visible. The multi view pose estimationmethod of FIG. 1 uses the visible elements in FIGS. 2, 3, and 4 asinputs.

FIG. 5 shows a schematic drawing of the structure of bronchial airwaysas utilized in the exemplary method of FIG. 1. The airway centerlinesare indicated by reference numeral 530. A catheter is inserted into theairways structure and imaged by a fluoroscopic device with an imageplane 540. The catheter projection on the image is illustrated by thecurve 550 and the radiopaque markers attached to it are projected ontopoints G and F.

FIG. 6 is an image of a bronchoscopic device tip attached to abronchoscope, in which the bronchoscope can be used in the exemplarymethod of FIG. 1.

FIG. 7 is an illustration according to an embodiment of the method ofthe present invention, where the illustration is of a fluoroscopic imageof a tracked scope (701) used in a bronchoscopic procedure with anoperational tool (702) that extends from it. The operational tool (702)may contain radio opaque markers or a unique pattern attached thereto.

FIG. 8 is an illustration of an exemplary embodiment, showing atwo-dimensional (“2D”) jig with a linear pattern of radiopaque markers.The rectangular area denoted by reference numeral 810 indicates aportion of the jig that may be captured in the field of view of afluoroscopic image. The portion of the pattern captured in the area 810is locally unique and distinct from any other portion of the pattern.

FIG. 9 is an illustration of an exemplary embodiment, showing a 2D jigwith a circular pattern of radiopaque markers. The rectangular areadenoted by reference numeral 910 illustrates a portion of the jig thatmay be captured in the field of view of a fluoroscopic image. Theportion of the pattern captured in the area 910 is locally unique anddistinct from any other portion of the pattern.

FIG. 10 is an illustration of an exemplary embodiment, showing athree-dimensional (“3D”) jig with a linear pattern of radiopaquemarkers. The radiopaque markers are placed in two different planesseparated by a distance H.

The figures constitute a part of this specification and includeillustrative embodiments of the present invention and illustrate variousobjects and features thereof. Further, the figures are not necessarilyto scale, some features may be exaggerated to show details of particularcomponents. In addition, any measurements, specifications and the likeshown in the figures are intended to be illustrative, and notrestrictive. Therefore, specific structural and functional detailsdisclosed herein are not to be interpreted as limiting, but merely as arepresentative basis for teaching one skilled in the art to variouslyemploy the present invention.

DETAILED DESCRIPTION

Among those benefits and improvements that have been disclosed, otherobjects and advantages of this invention will become apparent from thefollowing description taken in conjunction with the accompanyingfigures. Detailed embodiments of the present invention are disclosedherein; however, it is to be understood that the disclosed embodimentsare merely illustrative of the invention that may be embodied in variousforms. In addition, each of the examples given in connection with thevarious embodiments of the invention which are intended to beillustrative, and not restrictive.

Throughout the specification and claims, the following terms take themeanings explicitly associated herein, unless the context clearlydictates otherwise. The phrases “in one embodiment” and “in someembodiments” as used herein do not necessarily refer to the sameembodiments, though it may. Furthermore, the phrases “in anotherembodiment” and “in some other embodiments” as used herein do notnecessarily refer to a different embodiment, although it may. Thus, asdescribed below, various embodiments of the invention may be readilycombined, without departing from the scope or spirit of the invention.

As used herein, the term “based on” is not exclusive and allows forbeing based on additional factors not described, unless the contextclearly dictates otherwise. In addition, throughout the specification,the meaning of “a,” “an,” and “the” include plural references. Themeaning of “in” includes “in” and “on.”

As used herein, a “plurality” refers to more than one in number, e.g.,but not limited to, 2, 3, 4, 5, 6, 7, 8, 9, 10, etc. For example, aplurality of images can be 2 images, 3 images, 4 images, 5 images, 6images, 7 images, 8 images, 9 images, 10 images, etc.

As used herein, an “anatomical element” refers to a landmark, which canbe, e.g.: an area of interest, an incision point, a bifurcation, a bloodvessel, a bronchial airway, a rib or an organ.

As used herein, “geometrical constraints” or “geometric constraints” or“mutual constraints” or “mutual geometric constraints” refer to ageometrical relationship between physical organs (e.g., at least twophysical organs) in a subject's body which construct a similar geometricrelationship within the subject between ribs, the boundary of the body,etc. Such geometrical relationships, as being observed through differentimaging modalities, either remain unchanged or their relative movementcan be neglected or quantified.

As used herein, a “pose” refers to a set of six parameters thatdetermine a relative position and orientation of the intraoperativeimaging device source as a substitute to the optical camera device. As anon-limiting example, a pose can be obtained as a combination ofrelative movements between the device, patient bed, and the patient.Another non-limiting example of such movement is the rotation of theintraoperative imaging device combined with its movement around thestatic patient bed with static patient on the bed.

As used herein, a “position” refers to the location (that can bemeasured in any coordinate system such as x, y, and z Cartesiancoordinates) of any object, including an imaging device itself within a3D space.

As used herein, an “orientation” refers the angles of the intraoperativeimaging device. As non-limiting examples, the intraoperative imagingdevice can be oriented facing upwards, downwards, or laterally.

As used herein, a “pose estimation method” refers to a method toestimate the parameters of a camera associated with a second imagingmodality within the 3D space of the first imaging modality. Anon-limiting example of such a method is to obtain the parameters of theintraoperative fluoroscopic camera within the 3D space of a preoperativecomputed tomography (CT) image. A mathematical model uses such estimatedpose to project at least one 3D point inside of a preoperative CT imageto a corresponding 2D point inside the intraoperative X-ray image.

As used herein, a “multi view pose estimation method” refers a method toestimate to poses of at least two different poses of the intraoperativeimaging device. Where the imaging device acquires image from the samescene/subject.

As used herein, “relative angular difference” refers to the angulardifference of the between two poses of the imaging device caused bytheir relative angular movement.

As used herein, “relative pose difference” refers to both location andrelative angular difference between two poses of the imaging devicecaused by the relative spatial movement between the subject and theimaging device.

As used herein, “epipolar distance” refers to a measurement of thedistance between a point and the epipolar line of the same point inanother view. As used herein, an “epipolar line” refers to a calculationfrom an x, y vector or two-column matrix of a point or points in a view.

As used herein, a “similarity measure” refers to a real-valued functionthat quantifies the similarity between two objects.

In some embodiments, the present invention provides a method,comprising:

-   -   obtaining a first image from a first imaging modality,    -   extracting at least one element from the first image from the        first imaging modality,        -   wherein the at least one element comprises an airway, a            blood vessel, a body cavity, or any combination thereof;    -   obtaining, from a second imaging modality, at least (i) a first        image of a radiopaque instrument in a first pose and (ii) a        second image of the radiopaque instrument in a second pose,        -   wherein the radiopaque instrument is in a body cavity of a            patient;    -   generating at least two augmented bronchograms,        -   wherein a first augmented bronchogram corresponds to the            first image of the radiopaque instrument in the first pose,            and        -   wherein a second augmented bronchogram corresponds to the            second image of the radiopaque instrument in the second            pose,    -   determining mutual geometric constraints between:        -   (i) the first pose of the radiopaque instrument, and        -   (ii) the second pose of the radiopaque instrument,    -   estimating the first pose of the radiopaque instrument and the        second pose of the radiopaque instrument by comparing the first        pose of the radiopaque instrument and the second pose of the        radiopaque instrument to the first image of the first imaging        modality,        -   wherein the comparing is performed using:            -   (i) the first augmented bronchogram,            -   (ii) the second augmented bronchogram, and            -   (iii) the at least one element, and        -   wherein the estimated first pose of the radiopaque            instrument and the estimated second pose of the radiopaque            instrument meets the determined mutual geometric            constraints,    -   generating a third image; wherein the third image is an        augmented image derived from the second imaging modality which        highlights an area of interest,    -   wherein the area of interest is determined from data from the        first imaging modality.

In some embodiments, the at least one element from the first image fromthe first imaging modality further comprises a rib, a vertebra, adiaphragm, or any combination thereof. In some embodiments, the mutualgeometric constraints are generated by:

-   -   a. estimating a difference between (i) the first pose and (ii)        the second pose by comparing the first image of the radiopaque        instrument and the second image of the radiopaque instrument,        -   wherein the estimating is performed using a device            comprising a protractor, an accelerometer, a gyroscope, or            any combination thereof, and wherein the device is attached            to the second imaging modality;    -   b. extracting a plurality of image features to estimate a        relative pose change,        -   wherein the plurality of image features comprise anatomical            elements, non-anatomical elements, or any combination            thereof,        -   wherein the image features comprise: patches attached to a            patient, radiopaque markers positioned in a field of view of            the second imaging modality, or any combination thereof,        -   wherein the image features are visible on the first image of            the radiopaque instrument and the second image of the            radiopaque instrument;    -   c. estimating a difference between (i) the first pose and (ii)        the second pose by using a at least one camera,        -   wherein the camera comprises: a video camera, an infrared            camera, a depth camera, or any combination thereof,        -   wherein the camera is at a fixed location,        -   wherein the camera is configured to track at least one            feature,            -   wherein the at least one feature comprises: a marker                attached the patient, a marker attached to the second                imaging modality, or any combination thereof, and            -   tracking the at least one feature;    -   d. or any combination thereof.

In some embodiments, the method further comprises: tracking theradiopaque instrument for: identifying a trajectory, and using thetrajectory as a further geometric constraint, wherein the radiopaqueinstrument comprises an endoscope, an endo-bronchial tool, or a roboticarm.

In some embodiments, the present invention is a method, comprising:

-   -   generating a map of at least one body cavity of the patient,        -   wherein the map is generated using a first image from a            first imaging modality, obtaining, from a second imaging            modality, an image of a radiopaque instrument comprising at            least two attached markers,        -   wherein the at least two attached markers are separated by a            known distance, identifying a pose of the radiopaque            instrument from the second imaging modality relative to a            map of at least one body cavity of a patient,    -   identifying a first location of the first marker attached to the        radiopaque instrument on the second image from the second        imaging modality,    -   identifying a second location of the second marker attached to        the radiopaque instrument on the second image from the second        imaging modality, and    -   measuring a distance between the first location of the first        marker and the second location of the second marker,    -   projecting the known distance between the first marker and the        second marker,    -   comparing the measured distance with the projected known        distance between the first marker and the second marker to        identify a specific location of the radiopaque instrument inside        the at least one body cavity of the patient.

In some embodiments, the radiopaque instrument comprises an endoscope,an endo-bronchial tool, or a robotic arm.

In some embodiments, the method further comprises: identifying a depthof the radiopaque instrument by use of a trajectory of the radiopaqueinstrument.

In some embodiments, the first image from the first imaging modality isa pre-operative image. In some embodiments, the at least one image ofthe radiopaque instrument from the second imaging modality is anintra-operative image.

In some embodiments, the present invention is a method, comprising:

-   -   obtaining a first image from a first imaging modality,    -   extracting at least one element from the first image from the        first imaging modality,        -   wherein the at least one element comprises an airway, a            blood vessel, a body cavity or any combination thereof;    -   obtaining, from a second imaging modality, at least (i) a one        image of a radiopaque instrument and (ii) another image of the        radiopaque instrument in two different poses of second imaging        modality        -   wherein the first image of the radiopaque instrument is            captured at a first pose of second imaging modality,        -   wherein the second image of the radiopaque instrument is            captured at a second pose of second imaging modality, and        -   wherein the radiopaque instrument is in a body cavity of a            patient;    -   generating at least two augmented bronchograms correspondent to        each of two poses of the imaging device, wherein a first        augmented bronchogram derived from the first image of the        radiopaque instrument and the second augmented bronchogram        derived from the second image of the radiopaque instrument,    -   determining mutual geometric constraints between:        -   (i) the first pose of the second imaging modality, and        -   (ii) the second pose of the second imaging modality,    -   estimating the two poses of the second imaging modality        relatively to the first image of the first imaging modality,        using the correspondent augmented bronchogram images and at        least one element extracted from the first image of the first        imaging modality;        -   wherein the two estimated poses satisfy the mutual geometric            constrains.    -   generating a third image; wherein the third image is an        augmented image derived from the second imaging modality        highlighting the area of interest, based on data sourced from        the first imaging modality.

In some embodiments, anatomical elements such as: a rib, a vertebra, adiaphragm, or any combination thereof, are extracted from the firstimaging modality and from the second imaging modality.

In some embodiments, the mutual geometric constraints are generated by:

-   -   a. estimating a difference between (i) the first pose and (ii)        the second pose by comparing the first image of the radiopaque        instrument and the second image of the radiopaque instrument,        -   wherein the estimating is performed using a device            comprising a protractor, an accelerometer, a gyroscope, or            any combination thereof, and wherein the device is attached            to the second imaging modality;    -   b. extracting a plurality of image features to estimate a        relative pose change,        -   wherein the plurality of image features comprise anatomical            elements, non-anatomical elements, or any combination            thereof,        -   wherein the image features comprise: patches attached to a            patient, radiopaque markers positioned in a field of view of            the second imaging modality, or any combination thereof,        -   wherein the image features are visible on the first image of            the radiopaque instrument and the second image of the            radiopaque instrument;    -   c. estimate a difference between (i) the first pose and (ii) the        second pose by using a at least one camera,        -   wherein the camera comprises: a video camera, an infrared            camera, a depth camera, or any combination thereof,        -   wherein the camera is at a fixed location,        -   wherein the camera is configured to track at least one            feature,            -   wherein the at least one feature comprises: a marker                attached the patient, a marker attached to the second                imaging modality, or any combination thereof, and        -   tracking the at least one feature;    -   d. or any combination thereof.

In some embodiments, the method further comprises tracking theradiopaque instrument to identify a trajectory and using such trajectoryas additional geometric constrains, wherein the radiopaque instrumentcomprises an endoscope, an endo-bronchial tool, or a robotic arm.

In some embodiments, the present invention is a method to identify thetrue instrument location inside the patient, comprising:

-   -   using a map of at least one body cavity of a patient generated        from a first image of a first imaging modality,    -   obtaining, from a second imaging modality, an image of the        radiopaque instrument with at least two markers attached to it        and having the defined distance between them, that may be        perceived from the image as located in at least two different        body cavities inside the patient,    -   obtaining the pose of the second imaging modality relative to        the map    -   identifying a first location of the first marker attached to the        radiopaque instrument on the second image from the second        imaging modality,    -   identifying a second location of the second marker attached to        the radiopaque instrument on the second image from the second        imaging modality, and    -   measuring a distance between the first location of the first        marker and the second location of the second marker.    -   projecting the known distance between markers on each of the        perceived location of the radiopaque instrument using the pose        of the second imaging modality    -   comparing the measured distance to each of projected distances        between the two markers to identify the true instrument location        inside the body.

In some embodiments, the radiopaque instrument comprises an endoscope,an endo-bronchial tool, or a robotic arm.

In some embodiments, the method further comprises: identifying a depthof the radiopaque instrument by use of a trajectory of the radiopaqueinstrument.

In some embodiments, the first image from the first imaging modality isa pre-operative image. In some embodiments, the at least one image ofthe radiopaque instrument from the second imaging modality is anintra-operative image.

Multi View Pose Estimation

The application PCT/IB2015/000438 includes a description of a method toestimate the pose information (e.g., position, orientation) of afluoroscope device relative to a patient during an endoscopic procedure,and is herein incorporated by reference in its entirety.

The exemplary embodiments include a method which includes data extractedfrom a set of intra-operative images, where each of the images isacquired in at least one (e.g., 1, 2, 3, 4, etc) unknown pose obtainedfrom an imaging device. These images are used as input for the poseestimation method. As an exemplary embodiment, FIGS. 3, 4, 5, areexamples of a set of 3 Fluoroscopic images. The images in FIGS. 4 and 5were acquired in the same unknown pose while the image in FIG. 3 wasacquired in a different unknown pose. This set, for example, may or maynot contain additional known positional data related to the imagingdevice. For example, a set may contain positional data, such as C-armlocation and orientation, which can be provided by a Fluoroscope oracquired through a measurement device attached to the Fluoroscope, suchas protractor, accelerometer, gyroscope, etc.

In some embodiments, anatomical elements are extracted from additionalintraoperative images and these anatomical elements imply geometricalconstraints which can be introduced into the pose estimation method. Asa result, the number of elements extracted from a single intraoperativeimage can be reduced prior to using the pose estimation method.

In some embodiments, the multi view pose estimation method furtherincludes overlaying information sourced from a pre-operative modalityover any image from the set of intraoperative images. In someembodiments, a method for overlaying information includes: selecting, bya user, an area of interest on a preoperative image; generating a volumeof interest on the preoperative image; acquiring an intraoperative imageor video; calculating the pose of the intraoperative imaging modality;performing coarse registration between intraoperative images andpreoperative images; generating a set of features or patterns from avolume of interest of the preoperative image; implementing fineregistration to find the best fit between each of the features orpatterns; enhancing a signal matching pattern to highlight anatomy foundin the area of interest; and overlaying the signal sourcing from thereference image on the display/image. In some embodiments, a descriptionof overlaying information sourced from a pre-operative modality overintraoperative images can be found in International Patent ApplicationNo. PCT/IB2015/000438, which is incorporated herein by reference in itsentirety.

In some embodiments, the plurality of second imaging modalities allowfor changing a Fluoroscope pose relatively to the patient (e.g., but notlimited to, a rotation or linear movement of the Fluoroscope arm,patient bed rotation and movement, patient relative movement on the bed,or any combination of the above) to obtain the plurality of images,where the plurality of images are obtained from abovementioned relativeposes of the fluoroscopic source as any combination of rotational andlinear movement between the patient and Fluoroscopic device.

While a number of embodiments of the present invention have beendescribed, it is understood that these embodiments are illustrativeonly, and not restrictive, and that many modifications may becomeapparent to those of ordinary skill in the art. Further still, thevarious steps may be carried out in any desired order (and any desiredsteps may be added and/or any desired steps may be eliminated).

Reference is now made to the following examples, which together with theabove descriptions illustrate some embodiments of the invention in a nonlimiting fashion.

Example: Minimally Invasive Pulmonary Procedure

A non-limiting exemplary embodiment of the present invention can beapplied to a minimally invasive pulmonary procedure, whereendo-bronchial tools are inserted into bronchial airways of a patientthrough a working channel of the Bronchoscope (see FIG. 6). Prior tocommencing a diagnostic procedure, the physician performs a Setupprocess, where the physician places a catheter into several (e.g., 2, 3,4, etc.) bronchial airways around an area of interest. The Fluoroscopicimages are acquired for every location of the endo-bronchial catheter,as shown in FIGS. 2, 3, and 4.

After estimating the pose in the area of interest, pathways forinserting the bronchoscope can be identified on a pre-procedure imagingmodality, and can be marked by highlighting or overlaying informationfrom a pre-operative image over the intraoperative Fluoroscopic image.After navigating the endo-bronchial catheter to the area of interest,the physician can rotate, change the zoom level, or shift theFluoroscopic device for, e.g., verifying that the catheter is located inthe area of interest. Typically, such pose changes of the Fluoroscopicdevice, as illustrated by FIG. 4, would invalidate the previouslyestimated pose and require that the physician repeats the Setup process.However, since the catheter is already located inside the potential areaof interest, repeating the Setup process need not be performed.

FIG. 4 shows an exemplary embodiment of the present invention, showingthe pose of the Fluoroscope angle being estimated using anatomicalelements, which were extracted from FIGS. 2 and 3 (in which, e.g., FIGS.2 and 3 show images obtained from the initial Setup process and theadditional anatomical elements extracted from image, such as catheterlocation, ribs anatomy and body boundary). The pose can be changed by,for example, (1) moving the Fluoroscope (e.g., rotating the head aroundthe c-arm), (2) moving the Fluoroscope forward are backwards, oralternatively through the subject position change or either through thecombination of both etc. In addition, the mutual geometric constraintsbetween FIG. 2 and FIG. 4, such as positional data related to theimaging device, can be used in the estimation process.

FIG. 1 is a block diagram of an exemplary method, and shows thefollowing:

I. The component 120 extracts 3D anatomical elements, such as Bronchialairways, ribs, diaphragm, from the preoperative image, such as, but notlimited to, CT, magnetic resonance imaging (MRI), Positron emissiontomography—computed tomography (PET-CT), using an automatic orsemi-automatic segmentation process, or any combination thereof.Examples of automatic or semi-automatic segmentation processes aredescribed in “Three-dimensional Human Airway Segmentation Methods forClinical Virtual Bronchoscopy”, Atilla P. Kiraly, William E. Higgins,Geoffrey McLennan, Eric A. Hoffman, Joseph M. Reinhardt, which is herebyincorporated by reference in its entirety.

II. The component 130 extracts 2D anatomical elements (which are furthershown in FIG. 4, such as Bronchial airways 410, ribs 420, body boundary430 and diaphragm) from a set of intraoperative images, such as, but notlimited to, Fluoroscopic images, ultrasound images, etc.

III. The component 140 calculates the mutual constraints between eachsubset of the images in the set of intraoperative images, such asrelative angular difference, relative pose difference, epipolardistance, etc.

In another embodiment, the method includes estimating the mutualconstraints between each subset of the images in the set ofintraoperative images. Non-limiting examples of such methods are: (1)the use of a measurement device attached to the intraoperative imagingdevice to estimate a relative pose change between at least two poses ofa pair of fluoroscopic images. (2) The extraction of image features,such as anatomical elements or non-anatomical elements including, butnot limited to, patches (e.g., ECG patches) attached to a patient orradiopaque markers positioned inside the field of view of theintraoperative imaging device, that are visible on both images, andusing these features to estimate the relative pose change. (3) The useof a set of cameras, such as video camera, infrared camera, depthcamera, or any combination of those, attached to the specified locationin the procedure room, that tracks features, such as patches attached tothe patient or markers, markers attached to imaging device, etc. Bytracking such features the component can estimate the imaging devicerelative pose change.

IV. The component 150 matches the 3D element generated from preoperativeimage to their corresponding 2D elements generated from intraoperativeimage. For example, matching a given 2D Bronchial airway extracted fromFluoroscopic image to the set of 3D airways extracted from the CT image.

V. The component 170 estimates the poses for the each of the images inthe set of intra-operative images in the desired coordinate system, suchas preoperative image coordinate system, operation environment related,coordinated system formed by other imaging or navigation device, etc.

The inputs to this component are as follows:

-   -   3D anatomical elements extracted from the patient preoperative        image.    -   2D anatomical elements extracted from the set of intra-operative        images. As stated herein, the images in the set can be sourced        from the same or different imaging device poses.    -   Mutual constraints between each subset of the images in the set        of intraoperative images

The component 170 evaluates the pose for each image from the set ofintra-operative images such that:

-   -   The 2D extracted elements match the correspondent and projected        3D anatomical elements.    -   The mutual constraint conditions 140 apply for the estimated        poses.

To match the projected 3D elements, sourcing a preoperative image to thecorrespondent 2D elements from an inter-operative image, a similaritymeasure, such as a distance metric, is needed. Such a distance metricprovides a measure to assess the distances between the projected 3Delements and their correspondent 2D elements. For example, a Euclidiandistance between 2 polylines (e.g., connected sequence of line segmentscreated as a single object) can be used as a similarity measure between3D projected Bronchial airway sourcing pre-operative image to 2D airwayextracted from the intra-operative image.

Additionally, in an embodiment of the method of the present invention,the method includes estimating a set of poses that correspond to a setof intraoperative images by identifying such poses which optimize asimilarity measure, provided that the mutual constraints between thesubset of images from intraoperative image set are satisfied. Theoptimization of the similarity measure can be referred to as a LeastSquares problem and can be solved in several methods, e.g., (1) usingthe well-known bundle adjustment algorithm which implements an iterativeminimization method for pose estimation, and which is hereinincorporated by reference in its entirety: B. Triggs; P. McLauchlan; R.Hartley; A. Fitzgibbon (1999) “Bundle Adjustment—A Modern Synthesis”.ICCV 99: Proceedings of the International Workshop on Vision Algorithms.Springer-Verlag. pp. 298-372, and (2) using a grid search method to scanthe parameter space in search for optimal poses that optimize thesimilarity measure.

Markers

Radio-opaque markers can be placed in predefined locations on themedical instrument in order to recover 3D information about theinstrument position. Several pathways of 3D structures of intra-bodycavities, such as bronchial airways or blood vessels, can be projectedinto similar 2D curves on the intraoperative image. The 3D informationobtained with the markers may be used to differentiate between suchpathways.

In an exemplary embodiment of the present invention, as illustrated byFIG. 5, an instrument is imaged by an intraoperative device andprojected to the imaging plane 505. It is unknown whether the instrumentis placed inside pathway 520 or 525 since both pathways are projectedinto the same curve on the image plane 505. In order to differentiatebetween pathway 520 and 525, it is possible to use at least 2 radiopaquemarkers attached to the catheter having predefined distance “m” betweenthe markers. In FIG. 5, the markers observed on the preoperative imageare named “G” and “F”.

The differentiation process between 520 and 525 can be performed asfollows:

(1) Project point F from intraoperative image on the potentialcandidates of correspondent airways 520, 525 to obtain A and B points.

(2) Project point G from intraoperative image on the potentialcandidates of correspondent airways 520, 525 to obtain points C and D.

(3) Measure the distance between pairs of projected markers |AC| and|BD|.

(4) Compare the distances |AC| on 520 and |BD| on 525 to the distance mpredefined by tool manufacturer. Choose appropriate airway according toa distance similarity.

Tracked Scope

As non-limiting examples, methods to register a patient CT scan with aFluoroscopic device are disclosed herein. This method uses anatomicalelements detected both in the Fluoroscopic image and in the CT scan asan input to a pose estimation algorithm that produces a Fluoroscopicdevice Pose (e.g., orientation and position) with respect to the CTscan. The following extends this method by adding 3D space trajectories,corresponding to an endo-bronchial device position, to the inputs of theregistration method. These trajectories can be acquired by severalmeans, such as: attaching positional sensors along a scope or by using arobotic endoscopic arm. Such an endo-bronchial device will be referredfrom now on as Tracked Scope. The Tracked scope is used to guideoperational tools that extends from it to the target area (see FIG. 7).The diagnostic tools may be a catheter, forceps, needle, etc. Thefollowing describes how to use positional measurements acquired by theTracked scope to improve the accuracy and robustness of the registrationmethod shown herein.

In one embodiment, the registration between Tracked Scope trajectoriesand coordinate system of Fluoroscopic device is achieved throughpositioning of the Tracked Scope in various locations in space andapplying a standard pose estimation algorithm. See the following paperfor a reference to a pose estimation algorithm: F. Moreno-Noguer, V.Lepetit and P. Fua in the paper “EPnP: Efficient Perspective-n-PointCamera Pose Estimation”, which is hereby incorporated by reference inits entirety.

The pose estimation method disclosed herein is performed throughestimating a Pose in such way that selected elements in the CT scan areprojected on their corresponding elements in the fluoroscopic image. Inone embodiment of the current invention, adding the Tracked Scopetrajectories as an input to the pose estimation method extends thismethod. These trajectories can be transformed into the Fluoroscopicdevice coordinate system using the methods herein. Once transformed tothe Fluoroscopic device coordinate system, the trajectories serve asadditional constraints to the pose estimation method, since theestimated pose is constrained by the condition that the trajectoriesmust fit the bronchial airways segmented from the registered CT scan.

The Fluoroscopic device estimated Pose may be used to project anatomicalelements from the pre-operative CT to the Fluoroscopic live video inorder to guide an operational tool to a specified target inside thelung. Such anatomical elements may be, but are not limited to: a targetlesion, a pathway to the lesion, etc. The projected pathway to thetarget lesion provides the physician with only two-dimensionalinformation, resulting in a depth ambiguity, that is to say severalairways segmented on CT may correspond to the same projection on the 2DFluoroscopic image. It is important to correctly identify the bronchialairway on CT in which the operational tool is placed. One method used toreduce such ambiguity, described herein, is performed by usingradiopaque markers placed on the tool providing depth information. Inanother embodiment of the current invention, the Tracked scope may beused to reduce such ambiguity since it provides the 3D position insidethe bronchial airways. Having such approach applied to the brunchingbronchial tree, it allows eliminating the potential ambiguity optionsuntil the Tracked Scope tip 701 on FIG. 7. Assuming the operational tool702 on FIG. 7 does not have the 3D trajectory, although theabovementioned ambiguity may still happen for this portion 702 of thetool, such event is much less probable to occur. Therefore thisembodiment of current invention improves the ability of the methoddescribed herein to correctly identify the current tool's position.

A Jig and Methods of Using

An exemplary method to calculate mutual constraints between images byusing radiopaque markers positioned in the fluoroscopic device field ofview is described above. This section will describe methods to position,detect, and/or identify these markers in fluoroscopic images.

In some embodiments, radiopaque markers are attached to a rigid jig inpredetermined, fixed positions. In some embodiments, the radiopaquemarkers include a radiopaque metal. In some embodiments, the radiopaquemarkers include a radiopaque plastic. In some embodiments, theradiopaque markers include another radiopaque material. In someembodiments, the radiopaque markers have uniform shapes. In someembodiments, the shapes are spheres. In some embodiments, the shapes arerods. In some embodiments, the shapes are cubes. In some embodiments,the shapes are another uniform shape not mentioned herein. In someembodiments, the radiopaque markers are positioned on the jig. In someembodiments, the radiopaque markers are configured to be detected in afluoroscopic image using an image processing algorithm. In someembodiments, the image processing algorithm is a blob detectionalgorithm.

In some embodiments, the blob detection algorithm is a template matchingalgorithm. In some embodiments, a template of a blob is moved over thesearch image and blobs are detected where the template matches a part ofthe image. In some embodiments, a template matching algorithm includesthe following steps: 1. Overlay the template on the initial imageposition (0,0). 2. Calculate the sum of squared differences (SSD) or thesum of absolute differences (SAD) for the overlaid area and store it ina correlation matrix. 3. Move on to the next image position and repeatstep 2 until the final image position is reached. In some embodiments,bright spots in the correlation image correspond to probable bloblocations. In some embodiments, by defining a threshold, an exact numberof blobs and exact locations can be used as result. In some embodiments,when the template covers pixels outside the image, those values could becalculated by mirroring or extrapolation. In some embodiments, thetemplate positions could be restricted to positions with templatecoverage within the image. In some embodiments, small templates can beused to detect primitive blobs while large templates can detectspecifically shaped blobs. In some embodiments, to get a more flexibleblob detection, multiple templates could be designed.

In some embodiments, the blob detection algorithm is a watersheddetection algorithm. In some embodiments, the watershed method assumesan image to be grey value mountains and simulates the process of rainfalling onto the mountains, running down the mountain range andaccumulating in basins. In some embodiments, this process is repeateduntil all basins are filled and only the watersheds between differentbasins remain. In some embodiments, these watersheds correspond tobright blobs, whereas dark blobs can also be obtained by taking thegradient amplitude image. In some embodiments, this flooding process isperformed on the gradient image, i.e. the basins should emerge along theedges. Normally, this algorithm will lead to an oversegmentation of theimage, especially for noisy image material, e.g. medical CT data. Eitherthe image must be pre-processed or the regions must be merged on thebasis of a similarity criterion afterwards.

In some embodiments, the blob detection algorithm is a spoke filteralgorithm. In some embodiments, a spoke filter algorithm includes thefollowing steps: 1. Apply edge filters to extract local edge elements ofall (8) orientations. 2. Mark pixels as “interior”, which lie within acertain distance of an edge element on a line perpendicular to the edgetangent direction. 3. Mark spoke crossings as being interior pixelsmarked by edge elements of different orientations. 4. Mark blobs asbeing crossings marked by 6, 7 or all 8 directions. In some embodiments,by varying the distance, blobs of various sizes can be detected. In someembodiments, an intensity pyramid is defined as a set of fine to coarseresolution images. In some embodiments, at each level, the spoke filteris applied to detect blobs. In some embodiments, for each image in theintensity pyramid, the edge elements can be calculated and summed forall images. In some embodiments, for the summed gradient image, step 2to 4 of the spoke filter algorithm can be followed to detect blobs atmultiple scales.

In some embodiments, the blob detection algorithm is an automatic scaleselection algorithm. In some embodiments, an automatic scale selectionalgorithm operates based on the principle that, in the absence of otherevidence, assume that a scale level, at which some combination ofnormalized derivatives assumes a local maximum over scales, reflects thesize of the corresponding blob. In some embodiments, scale levels areobtained by Gaussian smoothing. In some embodiments, the Gaussianfunction meets the requirement that no details are generated whenresolution decreases and it provides simpler pictures at coarse scale.In some embodiments, combinations to be used as basic blob detectors inGaussian scale-space are Laplacian and the Monge-Ampere operator. Insome embodiments, the Laplacian operator is defined as the trace of theHessian matrix, which is the square matrix of second-order partialderivatives of the image function. In some embodiments, by multiplyingthe trace with a scale parameter, the Laplacian operator can be used todetect scale-space maxima. In some embodiments, the Monge-Ampereoperator is defined as the scale-normalized determinant of the Hessianmatrix. In some embodiments, the scale parameter is multiplied twice toobtain scale invariance. In some embodiments, maxima over scales have anice behavior under rescalings of the intensity pattern: if an image isrescaled with a constant factor, than the scale at which the maximum isassumed, will be multiplied with the same factor. In some embodiments,this guarantees that image operations transform with size variations. Insome embodiments, in practice, blobs may be detected at coarse scales,and the localization properties may not be the best. Therefore, in someembodiments, a coarse-to-fine approach is needed to compute moreaccurate localization estimates.

In some embodiments, the blob detection algorithm is a sub-pixel preciseblob detection algorithm. In some embodiments, a sub-pixel precise blobdetection algorithm operates according to the following steps: 1.Initialize the expected rectangle orientations of the shorter and largerside. 2. Calculate the Hessian matrix, the eigenvector and the directionof the rectangle's shorter side using Gaussian smoothing with an 1Dkernel in the expected orientation of the shorter side of the rectangle.3a. Compute the curvature maximum along the direction of the rectangle'sshorter side using the profile along the direction of the larger side.3b. Analyze the gradients of the used profile to determine bias andremove it. 4a. Compute the curvature maximum along the direction of therectangle's larger side using the profile along the direction of theshorter side. 4b. Analyze the gradients of the used profile to determinebias and remove it. 5. Reconstruct the rectangle's center point fromboth profiles. In some embodiments, this method provides a way toconstruct the boundary of the blob and approximate the boundary by anellipse. In some embodiments, for blob classification, this method canextract attributes like the blob's boundary length, area, geometricmoments and the parameters of the fitted ellipse.

In some embodiments, the blob detection algorithm is an effective maximaline detection algorithm. In some embodiments, an effective maxima linedetection algorithm is a method where connected curves of modulus maximaat different scales—called maxima lines—are effectively selected, todivide blobs from noise. In some embodiments, the selection of maximalines is performed by the following steps: 1. Compute the 2D Gaussianscale-space. 2. Compute modulus maxima at every scale. 3. Connectmodulus maxima in adjacent scales that are close to each other and havethe same sign (plus or minus) to obtain maxima lines. 4. Remove maximalines that consist of coefficients that increase on average when scaledecreases; they associate to noise. 5. Remove maxima lines that do notcross at least 5 integer scales; they associate to white noise. 6.Compute the global maximum for each maxima line and remove maxima lineswhich deviate at scales larger than the global maximum scale; theyassociate to blob structures outside the blob boundary. 7. Join maximalines that cross in scalespace; the blob location is given by the crosspoint and its characteristic scale by the median of the global maximumscales of all joined maxima lines.

In some embodiments, the blob detection algorithm is a confidencemeasurement algorithm. In some embodiments, a confidence measurementalgorithm operates as follows: 1. The image is first converted intochannel images using a set of windowed cosine kernel functions. 2. Foreach of the images, a low-pass pyramid is generated 3. Because thefilter sums up to 1, a threshold of 0.5 is used to obtain binaryconfidence values, resulting in a clustering pyramid. 4. The image ispruned by deleting similar clusters that lie on top of each other. 5.The pixels left in the pyramid are used as seeds for region growingresulting in a region image. 6. For all regions, the raw moments oforder 0 to 2 are computed to approximate blobs by ellipses. In someembodiments, this results in an image of ellipses with different sizesand orientations, overlapping each other.

In some embodiments, the jig includes a support structure (e.g., aboard, a box, etc.) having radiopaque markers positioned thereon in apattern. In some embodiments, the jig includes a substantiallytwo-dimensional object (e.g., a board) having radiopaque markerspositioned thereon in a two-dimensional pattern. In some embodiments,the jig includes a three-dimensional object (e.g., a box) havingradiopaque markers positioned therein in a three-dimensional space. Insome embodiments, the jig is configured to be positioned above apatient's bed. In some embodiments, the jig is configured to bepositioned below a patient's bed. In some embodiments, the jig isconfigured to be positioned between a patient's bed and a mattresspositioned thereon. In some embodiments, the jig is configured to beattached to or positioned on a patient's chest. In some embodiments, thejig is at least the size of a human chest. In some embodiments, the jigis larger than a human chest. In some embodiments, the pattern of theradiopaque markers attached to the jig is unique. As used herein, theterm “unique” means the pattern of radiopaque markers is non-repeatingand that any portion of the pattern of radiopaque markers (e.g., as maybe captured in a fluoroscopic image having a field of view encompassinga portion of a patient's body and a portion of the jig) is distinct fromany other portion of the pattern of radiopaque markers. In someembodiments, the pattern includes a set of lines as shown in FIG. 8, inwhich the radiopaque markers in each line form a unique pattern. In someembodiments, the pattern includes a set of concentric circles as shownin FIG. 9.

In some embodiments, the unique pattern is generated using linearfeedback shift registers (“LFSR”). In some embodiments, a Gold Code LFSRis used. In some embodiments, a Gold Code LFSR connects selected bitsfrom two feedback shift registers to the last produced code part withexclusive or (XOR) gates to the input of the next sequence. In someembodiments, the period of the code sequence depends on the registerlength N and the feedback pattern. In some embodiments, the maximumperiod is 2N−1. In some embodiments, to produce a Gold Code from anLFSR, the XOR connected bits are connected according to two principalpolynomials of the same order N. In some embodiments, five lines ofone-dimensional six-bit LFSR code are used. In some embodiments,two-dimensional LFSR code is used. In some embodiments, the uniquepattern is generated through the use of XOR cross connection of twoone-dimensional LFSR Gold-Codes with different principal polynomials ofthe same order and N×M register stages.

Consequently, in some embodiments, a user is able to identify a portionof a human's chest based on the location of the radiopaque markers onthe jig. In some embodiments, the jig is configured to be attached tothe patient's chest during the procedure, thereby allowing theestimation of the relative pose of the imaging device with respect tothe patient even if the patient moves during the procedure.

In some embodiments, when the radiopaque markers on the jig are imagedin different poses, the user can identify the poses of the markers withthe image(s) obtained to generate mutual constraints (e.g., component140 shown in FIG. 1). In some embodiments, this is accomplished bymatching each visible marker in the user's field of view with its mateon the jig (which has a known position due to the nature of the patternof radiopaque markers on the jig). In some embodiments, the correctpairing of visible markers can be achieved through designing thepositions of the radiopaque markers on the jig such that that in anyfield of view, the visible markers are projected as a unique pattern. Insome embodiments, by matching the visible unique pattern to thecorresponding unique pattern in the jig design, a correct pairing may beachieved since there is only one possible option of correspondence.

In some embodiments, the jig with patterned radiopaque markers is usedto determine the imaging device pose during an intervention procedure.For example, the section “Multi view pose estimation” above describes amethod to estimate imaging device poses for each image in a set ofimages. In some embodiments, provided that the jig is visible in eachimage, a user can estimate the jig's position in the coordinate systemof the imaging device. In some embodiments, the jig may not be attachedto the bed during pre-procedure imaging, such as, but not limited to, CTimaging. In such embodiments, after the jig position is calculated atleast once, this jig position may be used to determine theintra-operative imaging device pose during the procedure from a singleimage or a set of images (taken after the first calculation).

In some embodiments, the methods described herein can utilize the jig'sposition to determine the imaging device pose from a single view image.In some embodiments, a point-based pose estimation method (referred toherein as “method A”) is performed utilizing only the jig markersextracted from the intra-operative image as fiducial registration points(i.e., points that are used as a fixed basis of comparison). In someembodiments, a point-based pose estimation method is performed using acombination of anatomical elements (referred to herein “method B”) withthe jig markers serving as additional artificial fiducial registrationpoints for the pose estimation method. In some embodiments, method B isperformed according to the following: (1) identifying a plurality ofelements of a first image; (2) identifying a plurality of elements of asecond image (3) pairing the plurality of elements of the first image toa corresponding plurality of elements of the second image (or viceversa); (4) registering a plurality of elements of the first image tocorresponding pairs of the plurality of elements of the second image (orvice versa). In some embodiments, the registering is performed by fineand/or coarse registration. In some embodiments, method B is performedaccording to the following: (1) identifying a plurality (e.g., but notlimited to, 2, 3, 4, 5, 6, 7, 8, 9, 10, etc., elements) of elements(e.g., bronchi, ribs, etc.) from a first image (e.g., a CT image), (2)identifying a plurality of fluoroscopic elements on the first image(e.g., a CT image) and a plurality of fluoroscopic elements on thesecond image (e.g., a fluoroscopic image); (3) pairing a subset of theplurality of elements that correspond to elements (e.g., to bronchi,ribs, etc.) on the second image; and (4) registering the elements to thecorresponding pairs of the elements on the second image, where themapping results in a representation of the airway of the first image, orany combination thereof. In some embodiments, an image can be derivedfrom a raw image, e.g., but not limited to, a DDR image, an editedimage, a processed image, etc.

The point-based pose estimation method (method A) has the advantage ofbeing fast and robust since it relies on artificial markers introducedto the scene. Method B includes additional anatomical constraints ascompared to the point-based pose estimation method (method A), therebyincreasing registration accuracy. The accuracy of the point-based poseestimation method (method A) depends on the complexity of the jigdesign. Additional markers with a larger 3D spread (for example, alarger distance H in the jig shown in FIG. 10) result in increased poseestimation accuracy; therefore, a 3D jig (e.g., as shown in FIG. 10) mayprovide improved accuracy as compared to a 2D jig (e.g., the jigs shownin FIGS. 8 and 9). To overcome that, additional constraints may be addedsuch as the ones introduced by method B.

All publications, patents and sequence database entries mentioned hereinare hereby incorporated by reference in their entireties as if eachindividual publication or patent was specifically and individuallyindicated to be incorporated by reference.

While a number of embodiments of the present invention have beendescribed, it is understood that these embodiments are illustrativeonly, and not restrictive, and that many modifications may becomeapparent to those of ordinary skill in the art. The full scope of theinvention should be determined by reference to the claims, along withtheir full scope of equivalents, and the specification, along with suchvariations. Further still, the various steps may be carried out in anydesired order (and any desired steps may be added and/or any desiredsteps may be eliminated).

What is claimed is:
 1. A device for use during a medical imagingprocess, the device comprising: a support structure configured to bepositioned in proximity to at least a portion of a body of a patientduring the medical imaging process; and a plurality of radiopaquemarkers attached to the support structure, the plurality of radiopaquemarkers being positioned in a pattern such that an image capturing agiven portion of the pattern is unique from an image capturing any othergiven portion of the pattern.
 2. The device of claim 1, wherein theradiopaque markers include at least one of (a) a radiopaque metal or (b)a radiopaque plastic.
 3. The device of claim 1, wherein the radiopaquemarkers have a uniform shape.
 4. The device of claim 3, wherein theshape is selected from the group consisting sphere-shaped, rod-shaped,and cube-shaped.
 5. The device of claim 1, wherein the pattern includesone of (a) concentric circles or (b) a pattern of points arrayed alonglines.
 6. The device of claim 1, wherein the pattern is generated usinglinear feedback shift registers.
 7. The device of claim 6, wherein thepattern is generated using XOR cross connection of two one-dimensionallinear feedback shift register gold-codes with different principalpolynomials of the same order and NxM register stages.
 8. The device ofclaim 1, wherein the support structure is substantially planar.
 9. Thedevice of claim 1, wherein the support structure is three-dimensional.10. The device of claim 1, wherein the support structure is configuredto be positioned above a patient's bed.
 11. The device of claim 1,wherein the support structure is configured to be positioned below apatient's bed.
 12. The device of claim 1, wherein the support structureis configured to be positioned between a patient's bed and a mattresspositioned thereon.
 13. The device of claim 1, wherein the supportstructure is configured to be attached to a patient's chest.
 14. Thedevice of claim 1, wherein the support structure is at least the size ofa human chest.
 15. A method, comprising: providing a jig including asupport structure and a plurality of radiopaque markers attached to thesupport structure, the support structure configured to be positioned inproximity to at least a portion of a body of a patient during a medicalimaging process, the plurality of radiopaque markers being positioned ina pattern such that an image capturing a given portion of the pattern isunique from an image capturing any other given portion of the pattern;obtaining a first image from a first imaging modality; extracting atleast one element from the first image from the first imaging modality,wherein the at least one element comprises an airway, a blood vessel, abody cavity, or any combination thereof; obtaining, from a secondimaging modality, at least (i) a first image of the jig in a first poseof second imaging modality and (ii) a second image of the jig in asecond pose of second imaging modality, wherein the jig is positioned inproximity to a body of a patient; generating at least two augmentedbronchograms, wherein a first augmented bronchogram corresponds to thefirst image of the second imaging modality in the first pose, andwherein a second augmented bronchogram corresponds to the second imageof the second imaging modality in the second pose, determining mutualgeometric constraints between: (i) the first pose of the of secondimaging modality, and (ii) the second pose of the of second imagingmodality, estimating the first pose of the of second imaging modalityand the second pose of the of second imaging modality, wherein theestimation is performed using: (i) the first augmented bronchogram, (ii)the second augmented bronchogram, and (iii) the at least one element,and wherein the estimated first pose of the of second imaging modalityand the estimated second pose of the of second imaging modality meetsthe determined mutual geometric constraints, generating a third image;wherein the third image is an augmented image derived from the secondimaging modality which highlights an area of interest, wherein the areaof interest is determined from projecting data from the estimated firstpose and the estimated second pose.
 16. The method of claim 15, whereinthe mutual geometric constraints are generated by: a. estimating adifference between (i) the first pose and (ii) the second pose bycomparing the first image of the jig and the second image of the jig,wherein the estimating is performed using a device comprising aprotractor, an accelerometer, a gyroscope, or any combination thereof,and wherein the device is attached to the second imaging modality; b.extracting a plurality of image features to estimate a relative posechange, wherein the plurality of image features comprise anatomicalelements, non-anatomical elements, or any combination thereof, whereinthe image features comprise: patches attached to a patient, radiopaquemarkers positioned in a field of view of the second imaging modality, orany combination thereof, and wherein the image features are visible onthe first image of the radiopaque instrument and the second image of theradiopaque instrument; c. estimating a difference between (i) the firstpose and (ii) the second pose by using at least one camera, wherein thecamera comprises: a video camera, an infrared camera, a depth camera, orany combination thereof, wherein the camera is at a fixed location,wherein the camera is configured to track at least one feature, whereinthe at least one feature comprises: a marker attached the patient, amarker attached to the second imaging modality, or any combinationthereof, and tracking the at least one feature; d. or any combinationthereof.
 17. The method of claim 15, further comprising: tracking thejig for: identifying a trajectory, and using the trajectory as a furthergeometric constraint.